Unsupervised learning with stochastic gradient

A stochastic gradient is formulated based on deterministic gradient augmented with Cauchy simulated annealing capable to reach a global minimum with a convergence speed significantly faster when simulated annealing is used alone. In order to solve space-time variant inverse problems known as blind source separation, a novel Helmholtz free energy contrast function, H 1⁄4 E T0S; with imposed thermodynamics constraint at a constant temperature T0 was introduced generalizing the Shannon maximum entropy S of the closed systems to the open systems having non-zero input–output energy exchange E. Here, only the input data vector was known while source vector and mixing matrix were unknown. A stochastic gradient was successfully applied to solve inverse space-variant imaging problems on a concurrent pixel-bypixel basis with the unknown mixing matrix (imaging point spread function) varying from pixel to pixel. Published by Elsevier B.V.